High-Frequency Systolic Array-Based Transformer Accelerator on Field Programmable Gate Arrays

The systolic array is frequently used in accelerators for neural networks, including Transformer models that have recently achieved remarkable progress natural language processing (NLP) and machine translation. Due to the constraints of FPGA EDA (Field Programmable Gate Array Electronic Design Automation) tools limitations design methodology, existing deployment often cannot achieve high frequency. In this work, we propose a well-designed high-frequency an FPGA-based accelerator, which capable performing Multi-Head Attention (MHA) block position-wise Feed-Forward Network (FFN) block, reaching 588 MHz 474 different size, achieving frequency improvement 1.8× 1.5× on Xilinx ZCU102 board, while drastically saving resources compared similar recent works pushing utilization each DSP slice higher level. We also semi-automatic flow with constraint-generating as general solution deployment.